Predictive Tool to Support Decision-Making about Management of Conflicts of Interest for Guideline Development: Derivation and Validation

Purpose: to develop a tool that predicts which individuals on a guideline panel have a conflict of interest (COI) that may require special management to minimize risk of bias.

Background: COI is pervasive in medicine and can reduce trust in clinical practice guidelines. Guideline developers such as the American Society of Hematology (ASH) use multiple strategies to manage COI, from least restrictive (disclosure) to more restrictive (recusal from voting on specific recommendations) to most restrictive (ineligibility to serve on the panel). We sought to develop a tool that predicts which individuals on a guideline panel have a COI that requires a more restrictive management strategy, i.e., recusal. The tool could be used to support decision-making by the guideline developers about the best management strategy.

Design: Cross-sectional study of individuals who served on 9 ASH panels that developed clinical practice guidelines on venous thromboembolism.

Methods: On appointment, all panelists (N=120) used a standard ASH form to disclose recent (past 2 years) and current financial and nonfinancial interests. During the guideline development process (approximately 1 year), the forms were updated with new disclosures. At a final in-person meeting, on a question by question basis, individuals were recused if they had a direct financial interest in any for-profit company with a product that could be affected by the recommendation. Judgment by ASH was necessary to decide if an interest was current, financial, direct, and specific to the recommendation. To develop a predictive tool, we totaled all interests disclosed on each panelist's form, yielding a simple total COI score (TCOIscore). We then employed a logistic regression model to assess how well the TCOIscore predicted ASH decisions to require recusal (for any guideline question). Data from the randomly selected 5 panels (n=67 panelists) were used to derive the predictive equation, which was subsequently validated in 4 remaining panels (n=51 panelists). We assessed the model's calibration and discrimination by performing Hosmer-Lemoshow (H-L) goodness-of-fit (GOF) test and receiver operator characteristic (ROC) analysis, respectively.

Results: Our model demonstrated high accuracy in both derivation and validation samples (AUC=84%), i.e., the model accurately discriminated individuals for whom recusal was required by ASH for at least one guideline question from individuals for whom recusal was not required. The model showed excellent calibration in both derivation sample (H-L GOF: p=0.93) and validation sample (H-L GOF GOF: p=0.33;), (Fig 1). The optimal TCOIscore to predict recusal appears to be ≥4. The panel members with TCOIscore ≤3 were never asked to be recused from voting, while those with TCOIscore ≥ 8 were always required to recuse from voting (at least for one guideline question).

Conclusions: Individuals who disclose ≥5 recent or current interests across multiple categories including both financial (e.g., equity, personal income, research funding) and nonfinancial (e.g., published opinions, clinical specialty) are most likely to have a COI that requires special management by a guideline developer. By predicting COI, this tool may support decision-making about best management and, in turn, minimize risk of actual or perceived bias that may affect guidelines recommendations.

Figure 1

View largeDownload slide

Figure 1

View largeDownload slide

Close modal